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Abstract. It is well known that quantum continuous observations and nonlin- 
ear filtering can be developed within the framework of the quantum stochastic 
calculus of Hudson-Parthasarathy. The addition of real-time feedback con- 
trol has been discussed by many authors, but the foundations of the theory 
still appear to be relatively undeveloped. Here we introduce the notion of a 
controlled quantum flow, where feedback is taken into account by allowing 
the coefficients of the quantum stochastic differential equation to be adapted 
processes in the observation algebra. We then prove a separation theorem for 
quantum control: the admissible control that minimizes a given cost function 
is a memoryless function of the filter, provided that the associated Bellman 
equation has a sufficiently regular solution. Along the way we obtain results 
on existence and uniqueness of the solutions of controlled quantum filtering 
equations and on the innovations problem in the quantum setting. 



1. Introduction 

Quantum feedback control is a branch of stochastic control theory that takes 
into account the inherent uncertainty in quantum systems. Though quantum 
stochastic control was first investigated in the 1980s in the pioneering papers of 
Belavkin jBel83[ IBel88| it is only recently that this has become a feasibl e technol- 
ogy, as demonstrated by recent laboratory experiments in quantum optics jAAS^02l 
IGSM04j . On the other hand, modern computing and sensing technology are rapidly 
reaching a level of miniaturization and sensitivity at which inherent quantum uncer- 
tainties can no longer be neglected. The development of control theoretic machinery 
for the design of devices that are robust in presence of quantum uncertainty could 
thus have important implications for a future generation of precision technology. 

Though not as mature as their classical counterparts, mathematical tools for 
quantum stochastic analysis have been extensively developed over the last two 
decades following the introduction of quantum stochastic calculus by Hudson and 
Parthasarathy |HP84| . Quantum stochastic differential equations are known to 
provide accurate Markov models of realistic quantum systems, particularly the 
atomic-optical systems used in quantum optics, and continuous-time optical mea- 
surements are also accurately described within this framework. Furthermore, non- 
linear filtering theory for quantum systems has been extensively developed |Bel92l 
IBGM04llBV05j and provides a suitable notion of conditioning for quantum systems. 
Nonetheless the theory of quantum control is still very much in its infancy, and de- 
spite the large body of literature on classical stochastic control only a few rigorous 
results are available in the quantum case. Our goal here is to make a first step in 
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this direction by proving a quantum version of a simple but important theorem in 
classical control theory: a separation theorem for optimal stochastic controls. 

To set the stage for the remainder of the article, let us demonstrate the idea 
with a simple (but important) example. We consider an atom in interaction with 
the vacuum electromagnetic field; the atom can emit photons into the field, and we 
allow ourselves to control the strength of a fixed atomic Hamiltonian. The system 
dynamics is given by the quantum stochastic differential equation (QSDE) 

djt{X) = u{t) 3t{i[H, X])dt + 3t{C[X]) dt + jt[[X, L])dAl + jt{[L\X]) dAt 

where jt{X) denotes the atomic observable X at time i, and for now uit) is a 
deterministic control function (i.e. an open loop control). If we perform homodyne 
detection in the field, we observe the stochastic process Yt given by 

dYt = jt{L + L*) dt + dA; + dAt. 

We now have a system-observation pair as in classical stochastic control. Inspired 
by results in classical control theory, we begin by finding a recursive equation for 
TTt{X) = P{jt{X)\yt), the conditional expectation of the atomic observable X at 
time t, given the observations Yg up to time t. One obtains the nonlinear filter 

dTTt{X) = u{t) Trt{i[H, X]) dt + 7rtiC[X])dt 

+ {nt{XL + L*X) - TTt{L + L*)nt{X)) {dYt - ML + L*) dt). 

A crucial property of the conditional expectation is that the expectation of T^t{X) 
equals the expectation of jt(X), i.e. f{-Kt{X)) = ¥{jt{X)). Suppose we pose as our 
control goal the preparation of an atomic state with particular properties, e.g. we 
wish to find a control u{t) such that after a long time P(jt(X)) = P/(X) for some 
target state P / . Then it is sufficient to design a control that obeys this property for 
the filter 7r((X). The advantage of using the filter is that TTt{X) is only a function 
of the observations, and hence is always accessible to us, unlike jt{X) which is not 
directly observable. This approach was taken e.g. in jVSMdSI IMV05|. 

We immediately run into technical problems, however, as we have assumed in 
the derivation of the filtering equation that u{t) is a deterministic function whereas 
state preparation generally requires the use of a feedback control. One can of 
course simply replace the deterministic function u{t) in the filter with some (feed- 
back) function of the observation history, which is the approach generally taken 
in the literature, but does this new equation actually correspond to the associated 
controlled quantum system? The first issue that we resolve in this paper is to show 
that if u{t) is replaced by some function of the observation history in the equation 
for jt(X), then the associated filtering equation corresponds precisely (as expected) 
to the open loop filtering equation obtained without feedback where the control u{t) 
is replaced by the same function of the observations. 

The separation of the feedback control strategy into a filtering step and a control 
step, as suggested above, is a desirable situation, as the filter can be calculated 
recursively and hence the control strategy is not difficult to implement. It is not 
obvious, however, that such a separation is always possible. Rather than taking 
state preparation as the control objective, consider the optimal control problem in 
which the control goal is to find a control strategy that minimizes a suitably chosen 
cost function. This is a common choice in control theory, and in general the cost 
function can even be expressed in terms of the nonlinear filter. However, it is not at 
all obvious that the optimal control at time t only depends on 7rt(X); in principle. 
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the control could depend on the entire past history of the filter or even on some 
aspect of the observation process that is not captured by the filtering equation! 
The implementation of such a control would be awkward, as it would require the 
controller to have sufficient memory to store the entire observations history and 
enough resources to calculate an appropriate functional thereof. 

Optimal control problems are often approached through the method of dynamic 
programming, which provides a candidate control strategy in separated form. We 
will show that if we can find a separated control strategy that satisfies the dynamic 
programming equations, then this strategy is indeed optimal even with respect to 
all non-separated strategies. Thus the fortunate conclusion is that even in the case 
of optimal controls we generally do not need to worry about non-separated control 
strategies. This establishes a foundation for the separation principle of quantum 
control, by which we mean that as a rule of thumb the design of quantum feedback 
controls can be reduced to a separate filtering step and a control step. 

On the technical side, our treatment of quantum filtering proceeds by means 
of the reference probability approach [BV05 which is inspired by the approach 
of Zakai |Zak69| in classical nonlinear filtering. Our treatment of the separation 
theorem is directly inspired by the classic papers of Wonham |Won68| and Segall 
. A fully technical account of the results in this article will be presented in 
, and we apologize to the reader for the liberally sprinkled references to that 
paper. Here we will mostly neglect domain issues and similar technicalities, while 
we focus our attention on demonstrating the results announced above. 

This paper is organized as follows. In section [3 we briefly recall some of the 
basic ideas of quantum probability theory, and we develop the reference probability 
approach to quantum filtering without feedback. In section |31 we introduce the 
notion of a controlled quantum flow and show that for such models the controlled 
fllter takes the expected form. In section 0] we convert the filtering equations into 
classical stochastic differential equations and study their sample path properties; as 
a corollary, we obtain some results on the innovations problem. Finally, in section 
l^lwe introduce the optimal control problem and prove a separation theorem. 

2. Quantum probability and filtering 

The purpose of this section is to briefly remind the reader of the basic ideas 
underlying quantum probability and filtering. For a more thorough introduction 
we refer to |BV05| and the references therein. 

A quantum probability space {A, P) consists of a von Neumann algebra A, de- 
fined on some underlying Hilbert space J^, and a normal state P. If A is commu- 
tative then this definition is essentially identical to the usual definition in classical 
probability theory: indeed, the spectral theorem then guarantees that for some 
measure space there exists a *-isomorphism i : A i°°(ri, E, /j,) such 

that V{A) = Ep(t(A)) for all A G A, where Ep denotes the expectation w.r.t. 
the probability measure P ^ fi} Thus any self-adjoint element of A represents a 
bounded random variable (observable). In quantum models A is noncommutative, 
but in each realization we are only allowed to measure a commuting set of observ- 
ables which generate a commutative von Neumann algebra C. Hence if we fix a set 



^We will also denote the * -isomorphism as i, : P) — * L°° (O, S, /i, P); this means that the 
null sets are quotiented w.r.t. the measure fi, whereas P <^ fi is the image of the state P. 
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of (commuting) observations to be performed in every realization of the experiment, 
then many computations can be reduced to classical probability theory. 

To define the notion of a conditional expectation in quantum probability, we now 
simply "pull back" the associated notion from classical probability theory: 

Definition 2.1 (Conditional expectation). Let (^, P) be a quantum probability 
space and C C ,4 be a commutative von Neumann algebra. Define the commutant 
C = {A C A : AC ^ CA'iC & C}. The map P(-|C) : C ^ C is called (a version of) 
the conditional expectation onto C if P(P(A|C)C) = P(AC) for all A G C, C G C. 

Let us clarify the statement that this is nothing more than a classical conditional 
expectation. Let yl G C be self-adjoint; then Ca = vN(v4,C), the von Neumann 
algebra generated by A and C, is a commutative algebra, and the spectral theorem 
gives a *-isomorphism la to some L°°{Q,a-,^At IJ'A,'P a)- But then we can sim- 
ply calculate the classical conditional expectation and pull it back to the algebra: 
¥{A\C) = L'^^{'Rp^{LA{A)\a{LA{C)))). This is in fact identical to Definition [Q and 
can be extended to any A by writing is as i? -I- iC where i?, C are self-adjoint. 

The definition we have given is less general than the usual definition |Tak71j . 
In particular, we only allow conditioning onto a commutative algebra C from its 
commutant C . For statistical inference purposes (filtering) this is in fact sufficient: 
we wish to condition on the set of measurements made in a single realization of 
an experiment, hence they must commute; and it only makes sense to condition 
observables that are compatible with the observations already made, otherwise the 
conditional statistics would not be detectable by any experiment. Definition l2.1l has 
the additional advantage that existence, uniqueness, and all the basic properties can 
be proved by elementary means |BV05j. 

We list some of the most important properties of the conditional expectation: 
it exists, is a.s. unique (the difference between any two versions is zero with unit 
probability), and satisfies the least-squares property |||A — P(A|C)||| < |||A — C|||, 
|||X||p = ¥'{X*X) for aU C G C. Moreover we have (up to a.s. equivalence) 
linearity, positivity, invariance of the state P(P(A|C)) — P(A), the module property 
¥{AB\C) = B¥{A\C) for B G C, the tower property P(P(A|S)1C) = ¥{A\C) ifC C B, 
etc. Finally, we have the following Bayes-type formula: 

Lemma 2.2 (Bayes formula |BV05) 'l. Let (^, P) be a quantum probability space 
and C <Z A be a commutative von Neumann algebra. Furthermore, let V G C 
be such that V*V > 0, P(V^*P^) = 1. Then we can define a new state on C by 
Q{A) = ¥{V*AV) and we have Q{X\C) = P{V*XV\C) /P{V*V\C) for X G C . 

Up to this point we have only dealt with bounded operators. In the framework of 
quantum stochastic calculus, however, we unavoidably have to deal with unbounded 
operators that are affiliated to, not elements of, the various algebras mentioned 
above. As announced in the introduction, we largely forgo this issue here and we 
claim that all the results above (and below) can be extended to a sufficiently large 
class of unbounded operators. We refer to |BV06| for a complete treatment. 

Let us now introduce a class of quantum models that we will consider in this 
paper. The model consists of an initial system, defined on a finite-dimensional 
Hilbert space J^, in interaction with an external (e.g. electromagnetic) field that 
lives on the usual Boson Fock space F = Fs(L^([0, T])). We will always work 
on a finite time horizon [0, T] and we have restricted ourselves for simplicity to 
a single channel in the field (the case of multiple channels presents no significant 
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complications; it can be treated in the same manner on a case by case basis |BV05| . 
A completely general theory can also be set up, e.g. |Bel92j . but the required 
notations seem unnecessarily complicated.) Throughout we place ourselves on the 
quantum probability space (^,P) where A = B^W, B = B{Jfo), W = B(T), and 
P = p (E) (j> for some state p on B and the vacuum state cj) on W. We will use the 
standard notation Tt] = Ts{L^{[0,t])), Wt] = B{Tt]), etc. For / e i^([0,T]) we 
denote by e(/) G T the corresponding exponential vector, by <i> = e(0) the vacuum 
vector, and by At, A^ and At the fundamental noises. The reader is referred to 
|HP84I lBia95l |Mey93[ IPar92j for background on quantum stochastic calculus. 

For the time being we will not consider feedback control — we extend to this case 
in section Without feedback, the interaction between the initial system and the 
field is given by the unitary solution of the Hudson-Parthasarathy QSDE 

Ut = I+ f LsUsdA*^ f L:SsUsdAs+ f {Ss~I)UsdAs- f {iHs + \L*Ls)Usds. 
Jo Jo Jo Jo 

Here Lt, St and Ht are bounded processes of operators in B, St is unitary and Ht 
is self-adjoint. Without external controls the processes will usually be chosen to 
be time-independent, or we can imagine that e.g. the Hamiltonian Ht = u{t)H is 
modulated by some deterministic (open loop) scalar control u(t). In addition we 
specify an output noise that will be measured in the field; it takes the general form 

Zt= f Xs dAs + f asdA*+[ a* dA^ 
Jo Jo Jo 

where A : [0,T] R and a : [0, T] — > C are bounded scalar functions. Together 
Ut and Zt provide a full description of a filtering problem: any initial system 
observable X £ B is given at time t by the flow jt{X) = UtXUt, whereas the 
observation process that appears on our detector is given by Yj = Ut ZtUt- Using 
the quantum Ito rules, we obtain the explicit expressions 

(2.1) djt{X) = jt{i[Ht,X] + L*tXLt - \{L*tLtX -f XL^Lt}) dt 

+ 3t{S*t[X, Lt])dAl+jt{[L*t,X]St)dAt+jt{S*tXSt-X)dAt, 



(2.2) dYt = Xt dAt + jt {St {at + XtLt)) dAl + jt ((«? + XtL*t)St) dAt 

+ jt {XtLlLt + a*tLt + ati;) dt. 

This "system-theoretic" description in terms of the system-observations pair (|2.1|) 
and (|2.2|l is closest in spirit to the usual description of filtering and control problems 
in classical control theory. We will not explicitly use this representation, however. 

We now turn to the filtering problem, i.e. the problem of finding an explicit 
representation for the conditional state nt{X) — P{jt{X)\yt), X £ B, where 3^t = 
vN(Fs : < s < i) is the von Neumann algebra generated by the observations up 
to time t. Before we can go down this road we must prove that 7rt(A) is in fact well 
defined according to Definition l2.1l This is guaranteed by the following proposition. 

Proposition 2.1 (Nondemolition property). The observation process Yt satisfies 
the self-nondemolition condition, i.e. yt is commutative for all t G [0,T], and is 
nondemolition with respect to the flow, i.e. jt{X) £ for all X E B and t G [0, T] . 
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Proof. Let Zt ~ vN(Zs : < s < t). We begin by showing that Zt is a commutative 
algebra for aU t e [0,T]. To this end, define 

Z{c, d) = f Cs dAs + [ ds dA*s + /" d* dAs 
Jo Jo Ja 

so that Zt ~ Z{Xx[o.t]T o:X[o.t])- Using the quantimi Ito rules, we obtain 
[Z{C,D),Z{c,d)] = 

/ {Csds-CsDs)dAl+ j {Dlcs-d*Cs)dAs+ j {D^ds ~ d^Ds) dt. 
Jo Jo Jo 

But then we obtain [Zt, Zf] = for aU t, t' e [0, T] by setting 

Cs = AsX[o,t](s), Cs = XsXlo,t']{s), Ds = asX[o.t]{^), 4 = asX[o,t'] («)■ 
We conclude that the process Zt generates a commutative algebra. 

Next, we claim that U*ZsUs = UtZsUt for all s < t e [0,T]. To see this, let 
E G Zs he an arbitrary projection operator in the range of the spectral measure of 
Zs- Using the quantum Ito formula, we obtain 



jt{E)^jsiE) + 



I ]MH.,E] + LIEL, - \{LIL,E + ELlL,}) da 

J S 



+ MS;[E,L,])dA*,+ j,([Ll,E]S,)dA, + j,{S;ES, - E) dA, 

where jt{X) — UtXUt- But by construction E commutes with all H,, L,, S,, 
hence we obtain jt{E) — js{E). As this holds for all spectral projections E oi Zs, 
the assertion follows. We conclude that Yt = U^ZtUx for all t G [0,r]. But then 
yt = U^ZtUr, and as Zt is commutative yt must be as well. 

It remains to prove the nondemolition condition. To this end, note that jt{X) G 
UtBUt and yt = Ut ZtUt- But Zt consists of elements in W, which clearly commute 
with every element in B. The result follows immediately. □ 

To obtain an explicit representation for the filtering equation we are inspired by 
the classical reference probability method of Zakai Zak69|. The idea of Zakai is 
to introduce a change of measure such that under the new (reference) measure the 
observation process is a martingale which is independent of the system observables, 
which significantly simplifies the calculation of conditional expectations. The Bayes 
formula then relates the conditional expectations under the reference measure and 
the actual measure. We will follow a similar route and make use of the quantum 
Bayes formula of Lemma 12.21 The main difficulty is the choice of an appropriate 
change of measure operator V . 

In the classical reference probability method the change of measure is obtained 
from Girsanov's theorem. Unfortunately, there is no satisfactory noncommutative 
analog of the Girsanov theorem; even though Girsanov-like expressions can be 
obtained, they do not give rise to a change of state that lies in the commutant 
of the observations as required by Lemma 12.21 A different naive choice would be 
something like = fiUxXU^), so that Yt under R has the same statistics 

as the martingale Zt under P, but once again Ut does not commute with the 
observations. However, the latter idea (in a slightly modified form) can be "fixed" 
to work: starting from a change of state that is the solution of a QSDE, we can 
modify the QSDE somewhat so that the resulting solution still defines the same 
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state but has the desired properties. This trick appears to have originated in a 
paper by Holevo jHol91| . We state it here in the following form. 

Lemma 2.3. Let Ct,Dt,Ft,Gt,Ct,Ft be hounded processes, and let 
dVt = {Ct dAt + Dt dAl + Ft dAt + Gt dt}Vu 
dVt = {Ct dAt + Dt dA*t + Ft dAt + Gtdt}Vt, Vo^Vq. 

Then F{Vt*XVt) = V{Vt*XVt) for allX eB®W. 

Proof. As any state p on ;B is a convex combination of vector states, it is sufficient 
to prove the Lemma for any vector state p{B) = {v, Bv), v G J^q. Hence it suffices 
to prove that {Vt w ® XVt w (g) = (Vi w (g) XVt w ® $) for any X e B'g)W and 
V g J^Q, as P = p (g) 0. But clearly this would be implied by Vt w (g $ = Vt « g" 
Vv e J^o- Let us prove that this is in fact the case. 

As all the coefficients of the QSDE for Vt, Vt are bounded processes both Vf and 
Vt have unique solutions. Consider the quantity 

||(V^ -t4)i;®$f = {{Vt-Vt)v®^,{Vt-Vt)v(S<^>). 

Using the quantum Ito rule we obtain (see e.g. |Mey93| ) 

WiVt -Vt)v(^$ $f = / {{Vs ~Vs)v(E) $, {Gs + G:){V, -V)v® $) ds 
Jo 

+ [ {Ds{Vs -Vs)v® Ds{Vs -Vs)v® $) ds. 
Note that the last integrand can be expressed as 



\\DsiVs-Vs)v®<i>\\^ < 



sup II A I 
te[Q,T] 



\\{V,-Vs)v®^f 



To deal with the first integrand, note that Gg + G* are self-adjoint bounded oper- 
ators. Denote by Gf the positive part of Gg + G*, and by A'+ the square root of 
Gt (i.e. Gt = Kt*Kt). Then 

{{Vs -V,)v® $, (G, + GDiV - V;) « $) < \\KtiVs -V)v(E) $f . 

But as Gs -f G* is a bounded process, so is Kt and we have 



\\Kt{V-V)v®^f < 
Thus we obtain 



sup \\Kt 

te[o,T] 



+ ||2 



\\{Vs-Vs)v®<^>\\' 



\\{Vt~Vt)v®'S>W' <G / \\iV-Vs)v(®'S>\\^ds 



were by boundcdncss of Dt and 



G^ sup ||-ft:t+f+ sup llAf <oo. 
te[Q,T] te[o,T] 



But then by Gronwall's lemma ||(Vt — Vt) vCg$|| = 0, and the Lemma is proved. □ 
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We are now in the position to prove the main fihering theorem: a quantum 
version of the Kallianpur-Striebel formula. We consider separately two versions of 
the theorem for diffusive and counting observations, respectively (but note that the 
former also covers some cases with counting observations if At ^ 0.) 

Theorem 2.4 (Kallianpur-Striebel formula, diffusive case). Suppose that \at\ > 
for all t e [0, T] . Let Vt be the solution of the QSDE 

Vt = I+ f a-^LsVs{\sdAs + asdA:+a*dAs)- f {tH, + ^L^L^Ws ds. 
Jo Jo 

Then Vt is affiliated to Z[ for every t £ [0, T], and we have the representation 

TTtiX) = at{X)/at{I) with at{X) = U*1P{V* XVt\Zt)Ut, X € B. 

Theorem 2.5 (Kallianpur-Striebel formula, counting case). Suppose that \Xt \ > 
for all t G [0, T] . Let Et he the solution of the QSDE 



Et = I- 



f X-\l~as)E,dA:^ f X;'il-a:)EsdA,^l- f X;^\l - a,\^ E, ds 
Jo Jo ^ Jo 



and let Ct = E^ZtEt, Ct = vN(Cs : < s < t) = E;ZtEt, so that 

Ct^A;+At+ f XsdAs+ f A;i(l-|a,nds. 
^0 Jo 

Define Vt as the solution of the QSDE 

Vt = I+ f {Ls-X;^il~as))VsiXsdA, + dA*+dAs) 
Jo 

nt 



I {iHs + \LlLs + iA;"|l - a.p - X-\l - al)Ls)Vs ds. 
Jo 



Then Vt is affiliated to C't for every t G [0,T], and we have the representation 
TTt{X) = <7t{X)/at[I) with <7t{X) = U;EtT{Vt*XVt\Ct)E;Ut, X eB. 



Proof of Theorem \2.4\ We begin by using a general transformation property of the 
quantum conditional expectation. Let ?7 be a unitary operator and define a new 
state Q(X) = F{U*XU). Then F{U*XU\U*CU) = U*QiX\C)U provided that C 
is commutative and X G C . The statement is easily verified by direct application 
of Definition 12.11 The reason we wish to perform such a transformation is that in 
order to apply Lemma 12.31 it will be more convenient if we condition with respect 
to Zt rather than yt. From this point on, we fix a t G [0, T] and define the new 
state Q{X) = V{Ut*XUt). Evidently TTt{X) = Ut*Q{X\Zt)Ut. 

We would now like to apply Lemma lT^ to Q{X\Zt). Note that by Lemma lT^ we 
obtain F{Vt*XVt) = P{U;XUt) = Q(X). Moreover V is affiliated to B (g> Zt C Z[, 
as it is defined by a QSDE which is integrated against Zt and has coefficients in B 
(this statement can be rigorously verified by an approximation argument.) Hence 
by Lemma we obtain Q(X|Zt) = "^{V^ XVt\Zt)IFiy*Vt\Zt). The statement of 
the Theorem follows immediately. □ 

Proof of Theorem \2.5l The proof proceeds along the same lines as the proof of 
Theorem 12.41 except that in order to apply Lemma [2.31 we must make sure that 
the coefficient in front of dA'^ in the output noise does not vanish. To this end 
we perform an additional rotation by Et] the expression for Ct is obtained by 
direct application of the quantum Ito rules. If we introduce the transformed state 
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Q{X) = ¥{U;EtXE;Ut), we obtain Trt{X) = EtQ{X\Ct)E;Ut iov X e B (we 
have used the fact that any such X commutes with Et.) It remains to notice that 
P(V(*XVt) = Q(^) by Lemma [2.31 and by apphcation of the quantum Ito rules to 
E^Ut- The statement of the Theorem follows from Lemma [2. 21 □ 

Now that we have the quantum Kallianpur-Striebel formulas, it is not difficult 
to obtain a recursive representation of the filtering equations. Let us demonstrate 
the procedure with Theorem 12.41 Using the quantum Ito rules, we can write 

V;XVt^X+ f V;ii[H,,X] + L^XL, ~ ^{LlLsX + XL:Ls})Vsds 
Jo 

+ I V;iaJ^XLs + a-^*L*X + ia*as)'^\sL*XLs)VsdZt 
Jo 

for every X G B (and we use the shortened notation dZt = Xs dAg+ag dA*+a* dAg.) 
Taking the conditional expectation of both sides, we obtain 

F{V;XVt-X\Zt)= f ¥iV:it[Hg,X]+L:XLg-^{L:LgX + XL:Ls})Vg\Zg)ds 
Jo 

+ [ PiV;{a;^XL, + a;^*LlX + ia:as)-^XgLlXLs)Vs\Zs)dZu 
Jo 

where the fact that we can pull the conditional expectation into the integrals can 
once again be verified by an approximation argument. It remains to rotate the 
expression by C/t; another application of the quantum Ito rules gives 

at{X)^¥{X)+ f as{i[Hs,X]+ L*XLs - ^{L*LsX + XL*L,})ds 
Jo 

+ [ ns{a-^XL, + a-^*L*X + {alas)-^XsLlXL,)dYt 
Jo 

where dYt is given by 12.2|l . This is the noncommutative counterpart of the linear 
filtering equation of classical nonlinear filtering theory. Applying a similar proce- 
dure to Theorem 12 . 51 vields a linear filtering equation for the counting case. 

3. Controlled quantum flows and controlled filtering 

In the previous section we considered a quantum system where the coefficients 
Ht,Lt,St were deterministic functions in B; for example, we considered the case 
where L, S were constant in time and where the Hamiltonian Ht = u(t)H was 
modulated by a deterministic control. This gives rise to a filtering equation, e.g. the 
linear filtering equation that propagates at{-), which also depends deterministically 
on the control u{t). In a feedback control scenario, however, we would like to adapt 
the controls in real time based on the observations that have been accumulated; 
i.e., we want to make u{t) a function of the observations Ys up to time t. In this 
section we introduce the notion of a controlled QSDE, and show that this gives rise 
to a controlled linear filtering equation of the same form as in the previous section. 

Definition 3.1 (Controlled quantum flow). Given 
(1) an output noise Zt of the form 

Zt= f Es dAs + f TsdA:+ [ T: dAs 
Jo Jo Jo 
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such that St, Tt are adapted and afhUated to Zt = vN(Zs : < s < for 
every t £ [0, T], Si is self-adjomt, and is a commutative algebra; and 
(2) a controlled Hudson-Parthasarathy equation 

Ut=I+ f L.UsdAl^ f L:SsUsdA,+ f (5, -/)[/, dA,- / {iH, + \LlLs)Usds 
Jo Jo Jo Jo 

where Ht,Lt, St are affihated to ;B ® for every t £ [0, T], 

the pair {jt,Yt), where jt(X) = U;XUt {X eB®Zt) and = U^ZfUt, is called a 

controlled quantum flow jt with observation process Yt. 

To use this definition we must impose sufficient regularity conditions on the 
various processes involved so that these are indeed well defined. From this point 
onward we assume that St, Tt, i/i, Lt, St are bounded measurable processes, i.e. 

sup ||St|| < oo, 1 1— > Sfi/) is measurable V-i/" € (g) T, 
te[o,T] 

and similarly for the other processes. Under such assumptions (essentially bounded 
control requirements) we can show p3V06 that Zt is well defined and that Ut has 
a unique unitary solution (see also [Mey93| iHol96[ RTLWOII for related results.) 

Before we discuss filtering in the context of Definition 13.11 let us clarify the 
significance of this definition. First, note that we can use the quantum Ito rules to 
obtain the system-observations pair 

(3.1) djtiX) =. jtit[Ht,X] + LlXLt ~ \{LlLtX + XLlLt}) dt 

+ jt{S*t[X,Lt])dAl+jt{[L*t,X]St)dAt+]t{SlXSt~X)dKt, 

(3.2) dYt=^tdkt+3t{S*t{Tt + ^tLt)) dA,* + jt ((T,* + StL^^t) dAt 

+ jt iEtL;Lt + T;Lt + TtL;) dt, 

which is simply the controlled counterpart of 1)2.1(1 . (|2.2|) . The essential thing to 
notice is that though the quantities Ht,Lt etc. that appear in the equation for 
the flow Ut are affiliated to the output noise Zt, the quantities that appear in the 
system-observations model are in fact of the form jt{Ht), etc., which are affiliated 
to the observations yt- Our model is extremely general and allows for any of the 
coefficients of the QSDE, and even the measurement performed in the field, to be 
adapted in real time based on the observed process. To illustrate the various types 
of control that are typically used, we give the following examples. 

Example 3.2 (Hamiltonian feedback). Consider the controlled quantum flow 

dUt = {LdAl - L* dAt - \L*Ldt - iut{Zs<t)H dt) Ut, Zt^ At+ A*. 

That is, we have chosen St = 0, flxed L,H £ B, H ~ H* , and ut{Zs<t) is a 
bounded (real) scalar function of the output noise up to time t. This gives the 
system-observation pair 

djt {X) = Jt ( [X, L] ) dA; + Jt ( [L* , X] ) dAt 

+ jt{L*XL ~ liL*LX + XL*L))dt + ut{Ys<t)jtii[H,X])dt 

and dYt = dAt + dA^ + jt{L + L*) dt, where we have pulled the control outside jt- 
This scenario corresponds to a fixed system-probe interaction, measurement and 
system Hamiltonian, where we allow ourselves to feed back some function of the 
observation history to modulate the strength of the Hamiltonian; see e.g. |VSMn5j . 
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Example 3.3 (Coherent feedback). The controlled quantum flow defined by 

dUt = ((£ + ut{Zs<t)I) dA; - (L* + utiZs<tyi) dAt 

~ + Ut{Zs<trutiZ,<t)I + 2ut{Z,<t)L*) dt) Ut, 

where Zt — At + A^, describes an initial system driven by a field in a coherent 
state, where we modulate the coherent state amplitude through the bounded (com- 
plex) control ut{Zs<t) |Bou04| . As in the previous example, the control becomes a 
function of the observations when we transform to the system-theoretic description. 

Example 3.4 (Adaptive measurement). In this scenario we choose an uncontrolled 
Hudson-Parthasarathy equation dUt = {L dA^ — L* dAt - \L*L dt - iH dt) Ut, but 
the measurement in the probe field is adapted in real time by 

dZt = e-™'(^=2')dA* + e*"'(^=2')dAt 

where Ut{Zs<t) is a real scalar control function. This gives rise to the observations 

jt{L) e*"'(^=£') + jt(L*)e-™'(^=^') 



dYt = dZt 



dt. 



Evidently the control determines which of the system observables L e™ 4" L* e" 
is detected in the probe. The possibility to adapt the measurement in real time is 
useful for the detection of quantities that are not descr ibed by a system observable, 
such as the phase of an optical pulse. See e.g. |Wis95l IAAS+02| . 

We now wish to solve the filtering problem for a controlled quantum flow. It 
turns out that when we use the reference probability method, very little changes 
in the procedure outlined above. In particular, the proofs of Proposition 12. II and 
Lemma lT^ extend readily to the controlled case, and it is straightforward to extend 
the proofs of Theorems 12 . 41 and 12 . 51 to prove the following statements. 

Theorem 3.5 (Kallianpur-Striebel formula, diffusive case). Suppose that Tj has a 
hounded inverse for all t G [0, T] . Let Vt be the solution of the QSDE 

Vt^I+ f (S, dAs + Ts dA; + T: dAs) T-^LsVs - f {iH, + ^L*Ls)Vs ds. 
Jo Jo 

Then Vt is affiliated to Z't for every t £ [0, T], and we have the representation 

Tit{X) = <Jt{X)/at{I) with at(X) = U;nVt* XVt\Zt)Ut, XeB(g>Zt. 

Theorem 3.6 (Kallianpur-Striebel formula, counting case). Suppose that Sf has a 
bounded inverse for all t € [0, T] . Let Et be the solution of the QSDE 

Et = I+ /*S;i(l-T,)£;,dA:- fE;'{l-T:)E,dA,-l fE:^\l-Ts\'E,ds 
Jo Jo ^ Jo 

where \X\^ = X*X. Let Ct = E^ZtEt, Ct = vN(C^ : < s < t) = E*ZtEt, so that 

Ct ^A*t+At+ f E^EsEs dAs + f e:e-\1 - T:T,)E, ds. 
Jo Jo 

Define Vt as the solution of the QSDE 

1+ f {E^E.Es dA, + dA* + dAs) e;{Ls - - Ts))EsVs 

Jo 

e:{iHs + \l:Ls + _ T,|2 - ~ t:)Ls)EsVs ds. 



Vt = 
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Then Vt is affiliated to C[ for every t G [0,T], and we have the representation 
^t{X) = <7t{X)/at[I) with <7t{X) = U;Et¥{Vt*E;XEtVt\Ct)E;Ut, X eB^Zt. 

We can now obtain controlled filtering equations for crt(-) in a recursive form. 
The following statements are readily verified using the quantum Ito rules. 

Corollary 3.7 (linear filtering equation, diffusive case). Suppose that Tj has a 
bounded inverse for all t e [0,T]. Then crt(-) of Theorem \&.5\ satisfies 

atiX)=V{X)+ f nMHs.X] + LlXL,-^{L*L,X + XLlL,})ds 
Jo 

+ / (j,{T-^XL, + T-^*LlX + {TlTsr^^sLlXLs)dY, 

for X G B, where dYt is given by and nt{X) = at{X)/(Tt{I) . 

Corollary 3.8 (linear filtering equation, counting case). Suppose that St has a 
bounded inverse for all t e [0,T]. Then at{-) of Theorem \3.6l satisfies 

at(X)=P(X)+ / as{i[Hs,X]+ L*XL, ~ ^^{L^LsX + XL:L,})ds 
Jo 

+ [ asiE,L*XLs + T*XL, + TsL*X -E-\l-T*T,)X)x 
Jo 

{dYs-js{E;\l-T:Ts))ds) 

for X ^ B, where dYt is given by and T^t{X) — a't{X)/at{I). 

What is the relation between the open loop filters of the previous section and the 
closed loop filters associated to a controlled quantum flow? Consider for example 
the case of Hamiltonian control where Zt = At+ , St = Lt = L € B \s constant 
and Ht — u{t)H with H G B. In open loop u(t) is a deterministic function and we 
saw in the previous section that this gives rise to the linear filtering equation 

datiX) = u{t) at{i[H, X]) dt+at{L* XL-\{L* LX+XL* L}) dt+at{XL+L*X) dYt. 

In closed loop the model is given by the controlled quantum flow of Example 13.21 
and by CoroUar v 13.71 we obtain the controlled linear filtering equation 

datiX) = uiYs<t)c7tii[H,X])dt 

+ at{L*XL - \{L*LX + XL*L}) dt + (Tt{XL + L*X) dYf 

Evidently we obtain the same filter for the closed loop controlled quantum flow as 
we would obtain by calculating the open loop filter and then substituting a feedback 
control for the deterministic function u(t). This is in fact a general property of con- 
trolled filtering equations, as can be seen directly from the statement of Corollaries 
13. 71 and 13. 81 Though this property is usually assumed to hold true in the literature, 
we see here that it follows from the definition of a controlled quantum flow. 

4. Sample path properties and the innovations problem 

In the remainder of this paper we will use explicitly the properties of quantum 
filtering equations in recursive form, as given e.g. in Corollaries 13 . 71 and 13.81 In the 
next section we will show that under suitable regularity conditions, the quantum 
optimal control problem is solved by a feedback control policy that at time t is only 
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a function of the normalized solution of the linear filtering equation at that time. 
In order for this to be sensible, we have to show that the controlled linear filtering 
equation, and in particular its normalized form, has a unique strong solution. The 
purpose of this section is to investigate these and related properties of the solutions 
of recursive quantum filters. 

The approach we will take is to convert the entire problem into one of classical 
stochastic analysis. Note that all the quantities that appear in the linear filtering 
equations of Corollaries 13 . 71 and 13 . 81 are adapted and affiliated to the commutative 
algebra yt- Thus, we may use the spectral theorem to map the filter onto a classical 
stochastic differential equation driven by the (classical) observations. This will 
allow us to manipulate the filter by using the Ito change of variables formula for 
jump-diffusions and puts at our disposal the full machinery of classical stochastic 
differential equations driven by semimartingales. 

We begin by proving the following Proposition. This property will be crucial in 
the proof of the separation theorem; at this point, however, we are mostly interested 
in the fact that as a consequence, the observation process Yt is a semimartingale. 

Proposition 4.1 (Innovations martingale). Let dZt = Sj dAt + Tj dA'l + Tj dAt 
as in Definition \3.1\ Define the innovations process 

'Zt = WtZtUt - f tts{EsL:Ls + T:Ls + TsL^ds. 
Jo 

Then Zt is a yt-martingale, i.e. for all s < t Cz [0,T] we have ¥{Zt\ys) = Zg. 

Proof. We need to prove that P(Zt-Zsl3^s) = for aU s < t S [0, T], or equivalently 
P{(Zt - 'Zs)K) = for all s < t G [0, T] and K e y^. The latter can be written as 

P{U*tZtUtK) - V{U:Z,UsK) = ^ P(7r,(S,L:i, + T:^, + TsL*)K) ds 

for alls <t e [0,T] and K G 3^^- Since y^ = U^Z^Ut for all s < < e [0,r], it is 
sufficient to show that 

nU*tZtCUt)-nu:ZsCU,)= nUt{^sKL,C + TlL,C + T,LlC)Ut)ds 

for aU .s < t e [0, T] and e Z,,. But this follows directly from □ 

The innovations process is the starting point for martingale-based approaches to 
(quantum) filtering |Bel92l IB(i!M04j . The idea there is to obtain a particular mar- 
tingale which is represented as a stochastic integral with respect to the innovations. 
The method is complicated, however, by what is known as the innovations problem: 
it is not clear a priori whether the observations and the innovations generate the 
same (tT-)algebras LSQj, which is a prerequisite for the martingale representation 
theorem. The problem is resolved using a method by Fujisaki-Kallianpur-Kunita, 
where the Girsanov theorem is used to prove a special martingale representation 
theorem with respect to the innovations |LS01| . In contrast, the reference prob- 
ability method is completely independent from the innovations problem. Though 
we will not need this fact to prove the separation theorem, we will see at the end 
of this section that the equivalence of the observations and innovations cr-algebras 
follows as a corollary from the existence and uniqueness theorems. 

To return to the task at hand, it is evident from Proposition 14. II that the obser- 
vation process Yt can be written as the sum of the innovations process, which is a 
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martingale, and a process of finite variation, both of which are affihated to the alge- 
bra generated by the observations. Hence if we map Yt to its classical counterpart 
through the spectral theorem, we obtain a classical semimartingale. 

Remark 4.1. For convenience, we will abuse our notation somewhat and denote by 
Yt, Zt^ TTt{X), crt{X) both the corresponding quantum processes and the associated 
classical processes obtained through the spectral theorem. By St,Tf we denote 
the classical processes obtained by applying the spectral theorem to jt(Sj) and 
jt(Tt), whereas Lt,Ht are [dimJ^ x dim J^)-matrix valued processes obtained 
by applying the spectral theorem to each matrix element of jt{Lt) and jt{Ht). 
Hence St,Tt are Yt-adapted bounded scalar processes, whereas jt{Lt), jt{Ht) are 
Yt-adapted bounded matrix- valued processes. 

We now map the linear filtering equation of Corollarv l3.7l to a classical stochastic 
differential equation. This gives 

at{X) = P{X)+ f aAt[Hs,X] + L*XL, - ^{L^LsX + XL*^Ls})ds 
Jo 

+ f as-{T;lxL,^ + t;1*l:_x + ii::_Ts-)-'E,^ L:_XL,_)dYs 

jQ 

and similarly, we obtain the classical equivalent of CoroUarv 13.81 

atiX) = V{X)+ f nMHs,X] + L^XL, - ^{L*LsX + XL*L,})ds 
Jo 

+ f as-{Es- L:_XLs- + i::_XLs- + Ts-L:_X - - T:_Ts-)X)x 

Jo 

(dn-s;i(i-t:t,)ds) 

where now the stochastic integrals are classical Ito integrals with respect to the 
semimartingale Yt |Pro04| (the fact that we can map a quantum Ito integral with 
respect to fundamental processes to a classical Ito integral with respect to a semi- 
martingale is verified by approximation.) As we are now dealing with stochastic 
processes on the level of sample paths, we have to choose a modification such that 
the processes are well defined — this is an issue that does not occur on the level of 
QSDEs. We will make the standard choice |Pro04| that all our (semi) martingales 
are cadlag, and include explicitly the left limits as- etc. to enforce causality. 

We are now ready to apply classical stochastic analysis to our quantum filtering 
equations. We begin by normalizing the equations using the classical Ito formula. 

Proposition 4.2 (nonlinear filtering equation, pure diffusion case). Suppose that 
'E.t — and Tt has a bounded inverse for all t G [0, T]. Then Trt{-) satisfies with 
respect to the semimartingale observations Yt the ltd equation 

7rtiX)^ViX)+ f TTs{i[Hs,X] + LlXLs- ^{LlLsX + XL*Ls})ds 
Jo 

+ [ {7rsiT-'XLs + T-'*L:X)-TTsiT;'Ls + T-'*L:)nsiX)}dZs 
Jo 

for X g B, where Zt is the innovations process of Proposition Furthermore, 
there is a {dimJffo x dim. Mq) -matrix process pt such that iTt{X) = Tr[Xpt] for all 
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X G B, which satisfies the classical ltd stochastic differential equation 
Pt^Po + J \^-i[Hs,Ps]+LsPsL*-^{LlLsPs + PsL*Ls)^ds 

+ {f;'L,Ps + T-^*Psl: - Tr[(t;iL, + t;i*L:)p,] p,} dZ,. 

Proposition 4.3 (nonlinear filtering equation, pure jump case). Suppose that 
has a hounded inverse for all t S [0,T]. Then Trt{-) satisfies with respect to the 
semimartingale observations Yt the ltd equation 

TTtiX) = P(X) + /* TTsm,, X] + LlXLs - \{LIL,X + XLIL,}) ds 
Jo 

, f ^,_((T,_+S,_£.-)*X(T,_ + S,_L,_)) 

^Jo 1 ..-((T._+-,_L._)*(T._+S._L._)) --s-{X)^-s-dZs 

for X G B, where Zt is the innovations process of Proposition \4-l\ Furthermore, 
there is a (dim x dim Ji^) -matrix process pt such that T^tiX) — Tr[X/9t] for all 
X Cz B, which satisfies the classical ltd stochastic differential equation 

Pt = Po + J ^-i[Hs,Ps]+LsPsL*- ^{LlLsPs + PsL*Ls)^ds 
{Ts- + Es-Ls-)ps-{Ts- + Es-Ls-Y 




Ps- > ^s- dZs 



Proof of Propositions \4. 1^ and \4.':^ In pure diffusion case (Proposition ^3 the ob- 
servation process Yt is a continuous semimartingale, and hence the normalization 
is easily verified by applying the Ito change of variables formula to the Kallianpur- 
Striebel formula T^t{X) = at{X)/at{I). In both cases, the conversion to the matrix 
form follows from the fact that by construction TTtiX) is a random linear functional 
on B, and hence there is a unique random matrix pt such that Trt{X) = TT[Xpt] for 
all X Cz B. The expressions given are easily seen to satisfy this requirement. 

It remains to normalize (Jt{X) of Proposition ^21 Here we use the Ito change of 
variables formula for Stieltjcs integrals, but the manipulations are somewhat more 
cumbersome than in the continuous case. We begin by noting that 



Yt^ f s;_i(dr, + s;it:t,ds) 

Jo 

is a pure jump process with jumps of unit magnitude. To see this, we calculate 

[Y,Y]t = Yt^-2 f Ys-dYs^Yt 
Jo 

by applying the quantum Ito rules to (|3.2|) and using the spectral theorem. The 
quadratic variation [l",l"]t is by construction an increasing process, so Yt is also 
increasing and hence of finite variation. But any adapted, cadlag finite variation 
process is a quadratic pure jump semimartingale |Pro04| . meaning that 

Yt = [Y,Y]t= ^ it-Ys^)^. 

0<s<t 
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As Yt is an increasing pure jump process and Ys — K,- ~ (K, — Kj-)^, we conclude 
that Yt is a pure jump process with unit magnitude jumps. We can now rewrite 
the hnear filtering equation for the pure jump case in terms of Yt. This gives 



atiX)^V{X) 



f aMHs,X]+L*XLs - \{LlL,X + XL*Ls})ds 

JQ 



+ f <Js-{{Ts- +Es-L,^)*X{i:s- +Es-Ls-) - X){dYs -E-^ ds) 
Jo 

where the integral over Yt is a simple Stieltjes integral. To normalize at{X), we 
first use the change of variables formula for finite variation processes |Pro04j 

fiVt)-f{Vo)^ f f'{V,^)dVs+ J2 {f(Vs)~fiVs-)-f'{Vs-){Vs~V,-)} 
■^0 0<s<t 

to calculate (Jt{I)^^. We obtain directly that 

a,{I) - as-{I) - {a,^{B:_Bs-) - as-{I)) {Y, - Ys-) 

where Bs ^ T s + SsL^. Similarly we have 

- a,_(/)-i - {as-{B:_Bs-)-^ - a,_(/)-i) - 

We can thus express the correction term in the change of variables formula as 



^ {<js{i)-'-as-{i)-'+<ys-{i)-\<ys{i)-<ys-{i))] = 

0<s<t 

f (7r,_(B:_B,_) + 7r,_(B:_B,_)-i - 2) as- (I)-' dYs 

JO 

and we obtain 

1 ^, f' ns{B*sBs)-l ^_^, , f' iis-{B*s-Bs-)-^~l ^-^ 

(Jt{I) =1+/ —TTT ^s ds+ — — dYs. 

Jo ^s{l) Jo (^s-{l) 

Finally, applying the integration by parts formula for Stieltjes integrals to the 
Kallianpur-Striebel formula T^t{X) — (Jt{X) / <Jt{I) gives 



TTt{X)=¥{X) 



[ TTs{i[Hs,X] + L*sXLs - ^{L*sLsX + XLlLs}) ds 
Jq 

and it remains to notice that Ej} dZg = dY — Sj^ 7rs(i?|i?s) ds. □ 

Now that we have obtained the filtering equations in their sample path form, let 
us study the question of existence and uniqueness of solutions. 

Proposition 4.4 (Existence and uniqueness). Let S^, and the matrix elements 
of Ls and Hs have cadlag sample paths. Then both the linear and nonlinear filtering 
equations have a unique strong solution with respect to Yt. Moreover, matrix pt is 
a.s. positive with unit trace, i.e. a density matrix, for every t. 
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Proof. As cti^X) is linear by construction, we can find a matrix process rt such that 
(7t{X) = Tr[X Tt]. As we did for pt, we obtain directly that Tt must satisfy 

Tt = po + j ^-i[Hs,Ts] + LsTsLl ~ ^{LlLsTs + TsLlLs)^ ds 

+ [f-'L,T, + f-'*T,L:}dYs, 
in the pure diffusion case, and in the pure jump case must satisfy 

Tt = po + J ^-i[Hs,Ts] + LsTsL* - ^{L'*LsTs + TsLlLs)^ ds 

+ J {(t,,_+S,_L,_)r,_(t,_+S,,_L,_)*-r,_}x 

s;i(dy,-s;i(i-t:t,)ds). 

Both these equations are finite-dimensional linear stochastic differential equations 
with cadlag coefficients, for which the existence of a unique strong solution is a 
standard result Pro04 . Thus the unique solution Tt of these equations must indeed 
satisfy crtiX) — Tr[XTt] (this need not be true if the solution were not unique — then 
it could be the case that only one of the solutions coincides with at{-).) 

To demonstrate the existence of a solution to the equations for pt, note that by 
construction a-t{X) is a positive map (we will always assume that po is chosen so 
that ao{X) — 7ro(X) = Tr[X/9o] is a state, i.e. po is a positive matrix with unit 
trace.) Hence by uniqueness Tt must be a positive matrix for all t. Moreover, as 
the linear solution map tq i— > Tt is a.s. invertible |Pro04| tj is for all t a.s. not the 
zero matrix. This means that for all t the process pt = rt/Tr[Tt] is well-defined and 
satisfies the nonlinear filtering equations for pt which we obtained previously. Hence 
we have explicitly constructed a solution to the equations for pf, and moreover pt is 
a.s. a positive, unit trace matrix for every t. Finally, from the Kallianpur-Striebel 
formula it is evident that TTt{X) — Tr[Xpt]. 

It remains to prove that there are no other solutions pt that satisfy the nonlinear 
filtering equations, i.e. that the solution pt constructed above is unique. To this 
end, suppose that there is a different solution pt with po — Po that also satisfies 
the nonlinear filtering equation (with respect to dYt). Define the process Ht as the 
unique strong solution of the linear equation 

Ut = l+ f Tr[(t7iZ, + f:'*L:)ps]Us dYs 
Jo 

in the pure diffusion case, and as the unique strong solution of 

nt = l + J {Tr[(t,_-l-S,_L,_)*(t,_+S,_Z,_)p,_]-l}n,_x 

EjlidY, - :fs)ds) 

in the pure jump case. Using the Ito rules, one can verify that TltPt satisfies the 
same equation as Tt. But r* is uniquely defined; hence we conclude that pt = 
(II(/Tr[Tt])pt. By taking the trace of the nonlinear filtering equations it is easily 
verified that Tr[pt] = Tr[pt] = 1 a.s. for all t, and hence a.s. pt — pt. □ 
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To conclude the section, we will now prove that the innovations problem can be 
solved for the class of systems that we have considered under some mild conditions 
on the controls. It is likely that the innovations problem can be solved under more 
general conditions; however, as we will not need this result in the following, we 
restrict ourselves to the following case for sake of demonstration. 

Proposition 4.5 (Innovations problem). Let Ts and the matrix elements of Ls 
and Hg be cddldg semimartingales that are adapted to the filtration generated by 
the innovations process. The the observations Yt and the innovations Zt generate 
the same a -algebras (and hence also the same von Neumann algebras.) 

Proof. First, note that we can restrict ourselves to the case of diffusive observations. 
In the case of counting observations the result is trivial: as Yt — Zt is a continuous 
process, Yt can be completely recovered as the discontinuous part of Zt- Moreover, 
Zt is Yt-nieasurable by construction. Hence Yt and Zt generate the same a-algebras 
and there is no innovations problem. 

Things are not so simple in the diffusive case. Denote by the cr-algebra 
generated by Yt up to time t, and similarly by Sf the cr-algebra generated by Zt 
up to time t. The inclusion C holds true by construction, so we are burdened 
with proving the opposite inclusion T,^ C Sf . This is essentially an issue of "causal 
invertibility" : given only the stochastic process Zt, can we find a map that recovers 
the process Yt in a causal manner? Clearly this would be the case |LS01| if the 
nonlinear filtering equation has a unique strong solution with respect to Zt; as we 
have already established that it has a unique strong solution with respect to Yt the 
two solutions must then coincide, after which we can recover Yt from the formula 
dYt — dZt -|-Tr[(T( Lt + TtLj )pf] dt. Our approach will be precisely to demonstrate 
the uniqueness of pt with respect to Zt. 

To this end, consider the diffusive nonlinear filtering equation for pt given in 
Proposition 14. 21 where we now consider it to be driven directly by the martingale 
Zt rather than by the observations Yt. Now introduce the following quantities: 
let Xt be a vector which contains as entries all the matrix elements of pt, Lt, Ht, 
and the process T^, and let Kt be a vector that contains as entries all the matrix 
elements of Lt, Ht, the process Tt, and Zt. Then Kt is a vector of semimartingales 
and we can rewrite the nonlinear filtering equation in the form 



for a suitably chosen matrix function /. By inspection, we see that f{X) is poly- 
nomial in the elements of X and hence / is a locally Lipschitz function. Thus there 
is a unique solution of the nonlinear filtering equation with respect to Kg up to an 
accessible explosion time ^ |Pro04| . By uniqueness, the nonlinear filtering solution 
Pt with respect to Zt must coincide with the solution with respect to Yt up to the 
explosion time C. This also implies that for all t < the solution with respect to 
Zt must be a positive unit trace matrix. But the set of all such matrices is compact, 
and hence the accessibility of ^ is violated unless C — oo a.s. We conclude that the 
unique solution pt with respect to Zt exists for all time. □ 
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5. A SEPARATION THEOREM 



We are now finally ready to consider the control problem for quantum diffusions; 
i.e., how do we choose the processes Lt,Ht, etc. in the controlled quantum flow to 
achieve a certain control goal? As we will be comparing different control strategies, 
we begin by introducing some notation which allow us to keep them apart. 

Definition 5.1 (Control strategy). A control strategy fi — {E,T , S, L, H) is a 
collection of processes EtTTt, St, Lt, Ht defined on [0, T] that satisfy the conditions 
of Definition IXTl Given /x, we denote by = ('EtT'^ti St, Lt, Ht) the controls at 
time t, by fit] = {^[Q^t],^[o,t], S[o^t], L[o^t], H[Q^t]) the control strategy on the interval 
[0, t], and similarly by the control strategy on the interval [t, T]. 

Each control strategy /i defines a different controlled quantum flow. To avoid 
confusion, we will label the various quantities that are derived from the controlled 
flow by the associated control strategy. For example, jt{X) and are the flow 
and observations process obtained under the control strategy ^, etc. 

Our next task is to specify the control goal. To this end, we introduce a cost 
function which quantifies how successful a certain control strategy is deemed to be. 
The best control strategy is the one that minimizes the cost. 

Definition 5.2 (Cost function). Let be a process of positive operators, possibly 
dependent on the control strategy fi, such that is affiliated to vN(/it,;B), the 
algebra generated by the initial system and the control strategy at time t, for each 
t G [0, T]. Let Ct G B. The total cost is defined by the functional 



Cf^ and Ct are called the running and terminal cost operators, respectively. 

Ultimately our goal will be to find, if possible, an optimal control fi* that mini- 
mizes the expected total cost P(J[/i]). Let us begin by converting the latter into a 
more useful form. Using the tower property of conditional expectations, we have 



where is the algebra generated by Yf up to time t (note that the conditional 
expectations are well defined as jt{Ct) is affiliated to jt{B) (E) J^f.) But then 



and we see that the expected cost can be calculated from the associated filtering 
equation only. As the filter lives entirely in the commutative algebra y!^ we can now 
proceed, as in the previous section, by converting the problem into a classical sto- 
chastic problem. To this end, we use the spectral theorem to map the commutative 
quantum probability space {y!^, P) to the classical space L°°(f2'', S^, z/^, P'^) (note 
that different control strategies may not give rise to commuting observations, and 
hence the classical space depends on /z.) Thus we obtain the classical expression 
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We will use the same notations as in the previous section for the classical stochastic 
processes associated to Lt,Ht, etc. In addition, we denote by T,'^ the cr-algebra 
generated by the observations y/* up to time t. 

We have now defined the cost as a classical functional for an arbitrary control 
strategy. In practice not every control policy is allowed, however. First, note that in 
practical control scenarios only a limited number of controls are physically available; 
e.g. in the example of Hamiltonian feedback Ht = u{t)H we can only modulate the 
strength u{t) of a fixed Hamiltonian iJ, and we certainly cannot independently con- 
trol every matrix element of St, Lt, etc. Moreover we have expressed the (classical) 
cost in terms of the filter state 7rf (•); hence we should impose sufficient regularity 
conditions on the controls so that we can unambiguously obtain the filtered esti- 
mate from the observations using the nonlinear filtering equation of the previous 
section. To this end we introduce an admissible subspace of control strategies that 
are realizable in the control scenario of interest, and we require the solution of the 
optimal control problem to be an admissible control. 

Definition 5.3 (Admissible controls). Define the admissible range C R x C x 
B X B X B ioi every t e [0, T]. A control strategy is admissible if /it G a-S. (in 
the sense that {Et,Tt, St, Lt, Ht) £ a.s.) for every t e [0, T] and ^ has cadlag 
sample paths. The set of all admissible control strategies is denoted by 

Remark 5.4. Note that in order to satisfy Definition l3.ll ^t should be chosen to 
be a bounded set such that the only admissible S are unitary matrices and the only 
admissible H are self- adjoint matrices. 

The optimal control problem is to find, if possible, an admissible control strategy 
II* that minimizes the expected total cost, i.e. to find a /i* G such that 

P(JK]) = minP(JM). 

In principle /i* could depend on the entire history of observations up to time t. This 
would be awkward, as it would mean that the controller should have enough memory 
to record the entire observation history and enough resources to calculate a (possibly 
extremely complicated) functional thereof. However, as the cost functional only 
depends on the filter, one could hope that fit would only depend on p^, the solution 
of the nonlinear filtering equation at time t. This is a much more desirable situation 
as Pt can be calculated recursively: hence we would not need to remember the 
previous observations, and the feedback at time t could be calculated simply by 
applying a measurable function to p^. A control strategy that separates into a 
filtering problem and a control map is called a separated strategy. 

Definition 5.5 (Separated controls). An admissible control strategy p G ^ is said 
to be separated if there exists for every t G [0, T] a measurable map : S{B) — > ^t 
such that Pt ~ u^{p^), where Pt is the matrix solution of the nonlinear filtering 
equation at time t and S{B) is the set of positive matrices in B with unit trace. 
The set of all separated admissible strategies is denoted by 

The main technique for solving optimal control problems in discrete time is 
dynamic programming, a recursive algorithm that operates backwards in time to 
construct an optimal control strategy. The infinitesimal form of the dynamic pro- 
gramming recursion. Bellman's functional equation, provides a candidate optimal 
control strategy in separated form. The goal of this section is to prove that if we 
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can find a separated strategy /j, G that satisfies Bellman's equation, then this 
strategy is indeed optimal with respect to all control strategies in i.e. even those 
that are not separated. This result is known as a separation theorem. In classical 
stochastic control this result was established for linear systems in a classic paper 
by Wonham |Won68j and for finite-state Markov processes by Segall |Seg77| . The 
proof of the quantum separation theorem below proceeds along the same lines. 

We begin by introducing the expected cost-to-go, i.e. the cost incurred on an 
interval [t, T] conditioned on the observations up to time t. 

Definition 5.6 (Expected cost-to-go). Given an admissible control strategy /i, the 
expected cost-to-go at time t is defined as the random variable 

Wm) = Ep. TT^iCj^) ds + 4(Ct) 

The basic idea behind dynamic programming is as follows. Regardless of what 
conditional state we have arrived at at time t, an optimal control strategy should 
be such that the expected cost incurred over the remainder of the control time is 
minimized; in essence, an optimal control should minimize the expected cost-to- 
go. This is Bellman's principle of optimality. To find a control that satisfies this 
requirement, one could proceed in discrete time by starting at the final time T, 
and then performing a recursion backwards in time such that at each time step 
the control is chosen to minimize the expected cost-to-go. We will not detail this 
procedure here; see e.g. |Kus71| . Taking the hmit as the time step goes to zero 
gives the infinitesimal form of this recursion, i.e. Bellman's functional equation 

min \^ {u)V{t, 9) + Ti[0C^]}, te[0,T], 6^5(8) 

subject to the terminal conditon V{T,d) = Tr[0 Ct] (recall that is affiliated to 
vN(/Zt,S), and hence = C^* can be considered a ,B- valued measurable function 
of pf) The value function V{t,9) essentially represents the expected cost-to-go 
conditioned on the event that the solution of the nonlinear filtering equation 
takes the value 6 at time t. If the minimum in Bellman's equation can be evaluated 
for all t and 9, then it defines a separated control strategy /i by 

<(6I) = 'Aigmm\^(u)V{t,9)+Ti[9C^]], t G [0,T], 9 G S{B). 

In these equations ^[u] denotes the infinitesimal generator of the matrix nonlinear 
filtering equation given the control u; i.e., it is the operator that satisfies for any 
admissible control strategy /i the Ito change of variables formula 

!(t, pt) = f{s, p^) + J' ||^(a, p^,) + [^{p.)f] {a, da + j'^ G^^(p^) dZ. 

where / is any sufficiently differentiable function [C"^ in the diffusive case, in 
the pure jump case.) The expression for ^{u) and C^'^ is standard |Pro04j and 
can be obtained directly from Propositions 14. 21 and 14.31 

Our brief discussion of dynamic programming is intended purely as a motivation 
for what follows. Even if we had given a rigorous description of the procedure, 
the solution of Bellman's equation can only give a candidate control strategy and 
one must still show that this control strategy is indeed optimal. Thus, rather than 
deriving Bellman's equation, we will now take it as our starting point and show that 
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if we can find a separated control that solves it, then this control is optimal with 
respect to all admissible controls (i.e. there does not exist an admissible control 
strategy that achieves a lower expected total cost.) 

Theorem 5.7 (Separation theorem). Suppose there exists a separated admissible 
control strategy p, G '^'^ and a function V : [0, T] x S{B) — > R such that 

(1) The function V is in the first variable and in the second variable 
(diffusive case), or in both variables (pure jump case). 

(2) For all t e [0,T] and 9 G S{B), the function V satisfies 

^(t,0) +^«(0))F(t,0) + Tr [ecf^'^] = 0. 

(3) For all t G [0,T], u e cind 6 G SiB), the function V satisfies 

^{t,e) + ^{u)v{t,e) + TT[ec]^]>o. 

(4) For all 9 G S{B) the function V satisfies the terminal condition 

V{T,9) ^ Ti[9Ct]. 

Then the separated strategy p is optimal in , i.e. P(J[/i]) = min^/g^^ P( J[/i']). 

Proof. We begin by showing that for the candidate optimal control p, the value 
function V{t, ) evaluated at the solution of the nonlinear filtering equation at 
time t equals the expected cost-to-go W[^](t). To this end, we substitute condition 
^ with 9 = p^ and the terminal condition Q into Definition 15. 61 This gives 

W[p]it) = Ep. (^ViT,p!^) - f^^{s,p^) + [Jf«(p^))y](s,p^)| ds 

The purpose of condition ^ is to ensure that we can apply the Ito change of 
variables formula to V{t,p^). This gives 

W[p]{t)=Ep, (v{t,p'^)+ rG^,^{p^_)dZ 



But by a fundamental property of stochastic integrals jPro04j . the stochastic in- 
tegral of the bounded process G^l (ft- ) against the square-integrable martingale 
Zt is itself a martingale. Hence the conditional expectation of the second term 
vanishes, and as p^ is Sj^-measurable we obtain immediately 

w[pm = v{t,p^). 

In particular, since = p, we find that V{0, p) equals the expected total cost 
(5.1) V{Q,p)=P{J[p]). 

To show that p is optimal, let i/; G be an arbitrary admissible control strategy. We 
follow essentially the same argument as before, but now in the opposite direction. 
Using the Ito change of variable formula, we can verify that 



Vit,pt) = Ep, { ViT,pt)- I {^is,pt) + [.^{^,)V]{s,pt)\ds 
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Using condition (0J and the inequality Q with 6 — pf and u = V't; we obtain 



T 



j:t]=wmt)- 



But Po ~ Po ~ Pi so together with Equation 15. 1|) we obtain the inequahty 

P(J[m]) = 1^(0,p) <P(J[7/^]) 
for any iJj G which is the desired result. □ 

Remark 5.8. In a detailed study of the optimal control problem, separation the- 
orems are only a first step. What we have not addressed here are conditions under 
which Bellman's equation can in fact be shown have a solution, nor have we dis- 
cussed conditions on the feedback function of a separated control strategy that 
guarantee that the closed- loop system is well-defined (i.e., that it gives rise to cadlag 
controls.) These issues have yet to be addressed in the quantum case. 

The simplicitly of the separation argument makes such an approach particularly 
powerful. The argument is ideally suited for quantum optimal control, as we only 
need to compare the solution y of a deterministic PDE to the solution of the 
controlled quantum filter separately for every control strategy. Hence we do not 
need to worry about the fact that different strategies give rise to diflFerent, mutually 
incompatible observation algebras — the corresponding filters are never compared 
directly. The argument is readily extended, without significant modifications, to a 
wide variety of optimal control problems: on a finite and infinite time horizon, with 
a free terminal time or with time-average costs (see e.g. I0SO5I 1. 

Another option is to rewrite the cost directly in terms of the linear filtering 
equation, for example in the diffusive case: 



F{J[p]) = P l£ UrCi'Ut dt + U^*CtU^ 




VrC^Vt dt + V^CtV^ / (Cn dt + at^iCr) 
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where M^(X) = ¥{Ui^XUi^*). Note that under M^, the observation process is 
a Wiener process. Hence we can now perform dynamic programming, and find a 
separation theorem, directly in terms of the linear filtering equation. This is par- 
ticularly useful, for example, in treating the risk-sensitive control problem [Jam05| . 

Finally, a class of interesting quantum control problems can be formulated using 
the theory of quantum stopping times |PS87| : this gives rise to optimal stopping 
problems and impulse control problems in the quantum context. Such control 
problems are explored in |Van06| using a similar argument to the one used above. 
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